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The  NIST  SQL  Database  Loader: 
STEP  Working  Form  to  SQL 

Deborah  A.  Nickerson 


Introduction 

This  paper  discusses  the  NIST  SQL  Database  Loader,  this  utility  takes  a  STEP  file  and 
populates  SQL  tables  with  instances  o'"  STEP  entities.  This  document  is  intended  as  a 
programmer  and  user’s  guide.  After  reading  Transiauri£  Exgrcss  to  SQL:  a  User's 
Guide,  users  may  skip  to  the  User’s  Manual  section  (section  5.0).  PrograrnnKni  should 
read  the  following  prerequisite  reading  materials  before  this  document;  The  NT  ST 
^.grjsiDiLfsg!rnJg,^,S  The  NIST  STEP  Working  Form  Programmer’s  Reference. 
Translation  of  Express  Schema  into  SOL,  and  Translating  Express  to  SQL:  a  User’s 
Guide.  In  addition,  objects  in  the  STEP  Working  Form  arc  tightly  coupled  with  the  Ex¬ 
press  Working  Form;  sotiK  familiarity  with  the  latter  will  be  assumed  (see  Fed-X:  The 
NIST  Express  Translator  and  NIST  Express  Working  Form  Programmer's  Reference). 

The  NIST  SQL  Database  Loader  is  implemented  in  ANSI  Standard  C  ( ANSI89b].  The 
Oracle  Pro*C  precompiler  is  used  to  embed  SQL  standard  queries  in  accordance  with 
the  ANSI  standard  for  Embedded  SQL  [ANSISOc]. 

Context 

The  Standard  for  the  Exchange  of  Product  Model  Data  (STEP)  is  an  emerging  standard 
for  the  interchange  of  product  data  between  various  veiKiors’  CAD/CAM  systems  and 
other  manufacturing-related  software.  A  National  PDES  Testbed  has  been  established 
at  the  National  Institute  of  Standards  and  Technology  to  provide  testing  and  validation 
facilities  for  the  emerging  standard.  The  Testbed  is  fiind^  by  the  CALS  (Ckimputer- 
aided  Acquisition  and  Logistic  Support)  program  of  the  Office  of  the  Secretary  of  De¬ 
fense.  As  part  of  the  testing  effenrt,  NIST  is  charged  with  providing  a  software  toolkit 
for  manipulating  STEP  data.  This  NIST  PDES  Toolkit  is  an  evolving,  research-oriented 
set  of  software  tools.  This  document  is  one  of  a  set  of  reports  which  describe  various 
aspects  of  the  Toolkit  An  overview  of  the  Toolkit  is  provided  in  An  Introduction  to  The 
NIST  PDES  ToolkiL  along  with  references  to  the  oAer  documents  in  the  set 

For  further  information  on  the  NIST  SQL  Database  Loader  or  other  consponents  of  the 
Toolkit,  or  to  obtain  a  copy  of  the  software,  use  the  attached  order  form. 

Architecture 

As  discussed  in  The  NIST  Working  Form  for  STEP.  STEPparse  consists  of  two  sepa¬ 
rate  passes:  parsing  and  output  generation.  During  the  first  pass,  the  Express  file  is 
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parsed,  ami  the  Express  WcHidng  Form  is  cieaied.  Using  ihc  Express  Working  Fesm. 
the  STEP  file  is  parsed,  and  an  instance  of  a  Product  Daw  Deftmeon  is  built  in  the 
STEP  Working  Form.  The  second  pass  is  the  linking  of  the  output  module,  in  this  case, 
ihc  NIST  SQL  Database  Loader,  The  NIST  SQL  Database  Lo^r  can  be  dynarmcally 
or  statically  linked.  Once  linked,  the  NIST  SQL  Database  Loader,  following  through 
the  Product  Data  instance  in  the  STEP  Working  Form,  loads  the  Product  mfor- 
mation  into  the  appropriate  database  tables. 

2.1  MakeHle 

The  makefile  used  by  the  NIST  SQL  Database  Loader  is  the  generic  makefile  used  by 
all  the  modules  that  form  the  Validation  Testing  System  Toolkit.  Hooks  for  the  Oracle 
PiD*C  precompiler  were  added. 

2.2  Database  Schema 

As  discussed  in  Translating  Express  IQ^QL:  a  User^s  Guide,  an  Express  schema  is 
mapped  into  a  relational  database  schema.  This  schema  creates  the  necessary  tables 
ne^d  by  each  entity  in  the  Express  schema  and  dictionary  tables  to  store  Express  se¬ 
mantic  consirun  information.  In  addition  to  the  above  dictionary  ubles,  the  NIST  SQL 
Database  Loader  uses  a  cross  referencing  table,  sysSentity  id_tabieid.  This  la- 
blcconsistsof  threccolumns;  entityid,  tabieid,  and  f  ileid, reprcsenung 
the  STEP  file  entity  IDofanobject.the  table  IDofanobject,  and  the  file  number 
the  entity  instance  is  in.  Figure  1  shows  an  example  STEP  file  objects  and  their  cor¬ 
responding  entries  in  the  sysSentityid_tabieid  table.  The  sysSentity- 
id_t  ableid  table  is  created  at  the  same  time  the  other  data  dictionary  tables  arc 
created,  but  is  only  populated  by  the  NIST  SQL  Database  Loader. 
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STEP  file  objects. 

@  1  =DlRECnON{..0..0..  1 .0000000308274466); 
<§)2=DIR£CnON(..l.0000000308274466,0..0.); 
@3=DIRECnON(..0..  1 .0000000308274466.0.); 
@4»CARTESIAN_POINT(.,0.20.428009033203 125.11 ,22300052 
64282227); 


Representation  ot  table  SYS$ENTITYID_TABLEID  in  database. 
SYSSENTTTYID.TABLEID 


ENTITYID 

TABLEID 

FILEID 

1 

DIRECTION  lOOOOOCXlO 

NULL 

2 

DIRECnONlOOOOOOOl 

NULL 

3 

DIRECnON!00000002 

NULL 

4 

CARTESIAN_P01NT!00000(X)3 

NULL 

Figure  1:  STEP  File  objects  &  SYSSENTTrYID.TABLEID  Table 


3  stepwf^sql  Internals 

The  NIST  SQL  Database  Loader’s  program  name  is  stepwf_sql.  As  with  other 
STEP  Working  Form  output  modules,  st  epwf_sqi  is  linked  to  STEPparsc  via  an 
entry__point  function  with  the  following  parameters:  product  and  f  i  le,  corre¬ 
sponding  to  an  instance  of  a  Product  Data  Defmiticm  in  the  STEP  Wwking  Form  and 
the  output  file  respectively.  Unlike  other  STEP  Working  Form  output  modules,  the  out¬ 
put  file  is  not  used  as  input  data,  rather,  it  is  used  for  debugging  and  iidbnnational  pur¬ 
poses  only. 

3.1  •t«pirf_8ql  Flow  Control 

Moving  through  the  product  structure,  stepwf_sql  extracts  one  by  one  the  ob¬ 
jects  contained  within  the  product  structure.  Each  new  object  signals  the  start 
of  a  new  SQL  ins^  statement  If  the  object  is  sharable  by  other  cb  jects,  an  entry 
is  made  into  the  sys$entityid_tableid  table.  The  attributes  of  the  object  are 
extracted,  and  each  attribute’s  type  is  checked. 

For  simple  (Express  base)  type  attributes,  such  as  integen,  the  attribute’s  value  is  ap¬ 
pended  to  the  current  object’s  dynamic  SQL  insert  satement 

For  entity  type  attributes,  the  attribute  is  determined  to  be  cither  embedded  or  not  em¬ 
bedded.  For  non-embedded  entity  attributes,  a  query  is  made  to  the  sys  $ent  i  t  y- 
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icl_tableid  tabic  for  the  table  ID  of  the  referenced  entity  ID;  the  table  ID  is 
appended  to  the  current  ob  ject  ‘s  dynamic  SQL  insert  statement.  For  embedded  entity 
attributes,  a  recursive  pnaccss  is  started  where  the  entity  attribute  is  created  as  a  new 
object. 

For  select  type  attributes,  the  value  is  retrieved,  then  its  type  is  tested.  If  the  value 's  type 
is  that  of  an  entity,  a  recursive  pnxress  is  siancd  treating  the  value  of  the  select  as  a  new 
object.  Currcndy.  the  Express  to  SQL  mapping,  described  in  Translating  Express  to 
SQL:  a  Uscr^s  Guide,  states  that  if  the  type  of  the  value  of  the  select  is  not  entity,  then, 
the  value  is  represented  as  a  string  and  appended  to  the  current  ob  ject 's  dynamic 
SQL  insert  statement. 

For  aggregate  type  attributes,  a  new  dynamic  SQL  insert  staicmcni  is  assembled  for 
each  clcnctent  of  the  aggregate.  If  the  elentent  of  the  aggregate  is  of  type  aggregate,  a 
recursive  approach  is  taken,  adding  on  subscripts  to  the  newly  created  dynamic  SQL 
insert  statement  until  a  simple  element  is  found.  Figure  2  shows  an  example  of  the 
B_SPLINE_CXJRVE  aggregate  attribute  KNOTS. 


STEP  file  object:  B_SPLINE_CURVE.  The  KNOTS  attribute  is  highlighted. 

@370=B_SPLINE_CURVE(,.4.8. 

(#310.#320.#325.#330.#340.#350.#360.#365.#368). 

.UNIFORM_KNOTS..3.(5.4.5),rf?.(?,/.0,2.0j. 

(1.0,1 .0, 1 .0,1.0, 1 .0, 1 .0, 1 .0. 1 .0, 1 .0).,.F.); 


B_SPLINE_CURVE  aggregate  attribute  KNOTS  table. 


B_SPLINE_CURVE#KNOTS 


ED 

SUBSCRIPT.! 

VALUE 

B_SPLINE_CURVE!0000038 1 

0 

0.0 

B_SPLINE_CURVE!0000038 1 

1 

1.0 

B_SPLINE_CURVE!(XX)0038 1 

2 

2.0 

Figure  2:  Aggregates 

Once  a  dynamic  SQL  statement  is  completely  assembled,  it  is  inserted  into  the  data¬ 
base;  this  is  done  until  the  product  is  completely  loaded. 


3.2  Working  Form  Routines 

stepwf_sql  extracts  the  product  information  in  the  STEP  Working  Form  through  a 
series  of  S'l  EP  Working  Form  access  functions;  these  are  listed  in  The  NIST  STEP 
Working  Form  Programmer’s  Referentx.  In  addition,  access  functions  to  the  Express 
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3.3 


3.4 


3.5 


4 


5 


TtieNISTS 


Working  Form  are  also  used;  these  can  be  found  in  The  hflST  Ext)ress  Working  Form 
Programmer’s  Reference. 

Embedded  Pro*C 

The  NIST  SQL  Database  Loader  runs  interactively  with  the  Oracle  database,  using  em¬ 
bedded  Pro*C  code.  Pro*C  code  is  used  throughout  stepwf  sql  to  connect  to  the 
user’s  database  account,  extract  dictionary  information,  and  of  course,  insert  the  pan 
data.  Dynamic  SQL  is  used  to  insen  the  different  S'TCP  file  entities  of  the  pan  into  their 
respective  tables. 

Table  ID  Assembly 

To  insure  that  each  table  ID  is  distinct,  the  following  method  for  creating  tabic  IDs  was 
devised.  Given  an  object's  Express  entity  name,  the  EXPR£.SSYS$NAMES  table  is 
queried  for  the  Oracle  entity  name.  An  exclamation  point  (!)  is  appended  to  the  Oracle 
entity  name,  and  finally  a  number  is  appended.  This  number  is  generated  by  an  Oracle 
sequence,  TABLE_SEQ;  this  sequence  is  created  upon  database  schema  creation. 

Error  Checking 

Extensive  error  checking  is  included  in  stepwf  sql.  Since  the  Express  and  STEP 
parsers  are  executed  before  the  NIST  SQL  Database  Loader,  any  errors  and/or  warnings 
in  the  Express  and  STEP  files  should  be  flagged  before  reaching  this  software.  On  the 
other  hand,  there  may  be  errors  in  the  database.  For  this  reason.  stepwf_sql  will  rc- 
pon  errors  it  finds  in  a  descriptive  manner,  for  example,  if  there  is  no  table  in  the  data¬ 
base  that  matches  the  object  to  be  inserted,  a  rollback  of  the  database  is  executed, 
and  a  descriptive  error  message  will  be  displayed  including  the  entity  ID  of  the  ob¬ 
ject  where  execution  failed.  In  addition,  the  output  file  generated  lists  all  the  error 
free  insens  made  into  the  database  before  rollback. 

Issues  of  Concern 

Currently,  stepwf_sql  has  the  ability  to  load  in  a  partial  STEP  file,  then  at  a  later 
point  in  time,  load  the  subsequent  data.  It  docs  not  have  the  ability  to  load  more  than 
one  part  in  the  database.  In  order  to  load  multiple  parts  into  the  same  database,  the  issue 
of  how  to  differentiate  parts  needs  to  be  resolved.  At  the  present  time,  the  PSCM  struc¬ 
tures  would  have  to  be  populated  to  differentiate  pans;  there  is  nothing  explicit  in  the 
STEP  file  that  differentiates  pans. 

User’s  Manual 

This  section  deals  with  the  “how  to”  of  database  loading.  It  assumes  that  the  user  has 
already  created  a  database  from  the  Express  schema;  look  at  the  Translarin£  Express  to 
SOL:  User’s  Guide  for  instructions. 
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5.1  Coinnmnd  Line 

The  cotnmand  line  for  stepwf_sql  is  as  follows: 

%  stepwf_sql  -e  express_schema . f ile  -s  step. file 
The  Express  schema  file  used  should  be  the  same  as  the  one  used  to  build  the  database 
schema,  and  the  STEP  file  to  be  loaded  should  match  the  schema  exactly,  i.c.  all  at¬ 
tributes  and  references  should  follow  the  structure  of  the  schema. 

First,  the  Express  file  is  parsed,  building  the  Express  Working  Form.  No  errors  of  det¬ 
rimental  importance  to  the  database  schema  should  be  reported  at  this  time;  these 
should  have  been  dealt  with  at  schema  creation. 

Second,  the  STEP  file  is  parsed,  building  the  STEP  Working  Form,  Any  errors  should 
be  noted  with  great  care.  The  error  messages  from  the  STEP  parser  are  self-explanatory. 
If  error  messages  arc  present  and  the  program  has  not  stopped  processing,  abon  the  pro¬ 
cess,  and  fix  the  errors.  stepwf_sql  will  have  to  be  restarted. 

If  no  error  mcssa'tcs  are  reported,  the  user  will  be  asked  for  their  account  user  name  and 
password.  They  will  also  be  asked  to  fill  in  an  output  file  nan^;  this  output  file  is  for 
informational  and  debugging  purposes  only.  Error  messages  after  this  point  indicate  a 
problem  with  the  user's  database.  These  messages  are  also  straightforward,  for  case  in 
debugging.  Once  the  errors  are  conccicd,  stepwf_sql  will  need  to  be  restarted; 
stepwf_sql  rollbacks  the  daubasc  to  the  state  prior  to  execution. 

5.2  Useful  Database  Utilities 

There  are  a  number  of  useful  database  roanipuJation  utilities  available;  descriptions  of 
these  can  be  found  in  The  PDES  Testbed  User's  Guide.  Of  these  utilities,  the  utility 
userdropdat  a  drops  all  the  data  contained  in  a  user’s  database.  As  mentioned  in  the 
above  “Issues  of  Concern’’  section,  the  database  can  only  hold  one  part  at  a  time.  Each 
time  the  user  wants  to  load  a  new  STEP  file,  the  user  must  clear  the  database  using  the 
userdropdata  utility. 

6  Conclusion 

The  NIST  SQL  Database  Loader  takes  a  STEP  file  and  Express  schema  as  input.  Ex¬ 
press  schema  information  and  STEP  file  data  are  loaded  into  nwmory  woridng  forms. 
Data  is  retrieved  from  the  memory  resident  working  forms  and  used  to  populate  SQL 
tables  with  instances  of  STEP  entities. 

For  further  information  on  the  STEP  Working  Form,  the  database  schema,  or  other 
components  of  the  Toolkit,  or  to  obtain  a  copy  of  the  software,  use  the  attached  order 
form. 
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